Where the Linked Dependence Assumption Fails and How to Move Beyond It

The linked dependence assumption, a widely used simpli cation in probabilistic retrieval, is brie y reviewed and its validity is investigated. We show that the linked dependence assumption is violated particularly if the query features are good discriminators between relevant and non-relevant documents. We then propose the manual concept generation method, which takes into account the dependence between di erent query concepts. For a limited set of routing queries, experiments show a signi cant improvement when compared with retrieval methods which do not take into account feature dependencies. We consider this method to be the rst successful step which progresses beyond the linked dependence assumption.